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Read Naturally 



Program description 



Read Naturally is designed to improve reading fluency using a 
combination of books, audio-tapes, and computer software. 

This program includes three main strategies: repeated reading 
of Engiish text for oral reading fluency development, teacher 
modeling of story reading, and systematic monitoring of student 
progress by teachers. Students work at a reading level appropri- 
ate for their achievement level, progress through the program at 



their own rate, and work, for the most part, on an independent 
basis. The Read Naturally strategy is designed to increase time 
spent reading by combining teacher modeling, repeated reading, 
and progress monitoring. Although the program was not origi- 
nally developed for English language learners (ELL), materials for 
these students are now available. 



Research One study of a modified version of Read Naturally met the What from five schools in central Texas and examined effects on 
Works Clearinghouse (WWC) evidence standards with reserva- students’ reading achievement.^ 
tions. This study included 60 ELL elementary school students 



Effectiveness 



Read Naturally was found to have no discernible effects on elementary school ELL students’ reading achievement. 



Reading 


Mathematics 


Engiish language 


achievement 


achievement 


development 



Rating of effectiveness 


No discernible effects 


Not reported 


Not reported 


Improvement index^ 


Average: 0 percentile points 


Not reported 


Not reported 



Range: -5 to +6 percentile points 



1. The evidence presented in this report is based on avaiiabie research. Findings and conciusions may change as new research becomes avaiiabie. 

2. These numbers show the average and range of improvement indices for aii findings across the study. 
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Developer and contact 

Developed by Candyce Ihnot. Read Naturally, 750 S. Plaza Dr. 
#100, Saint Paul, MN 55120. Web: w\A/\A/.readnaturally.com. 
Email: info@readnaturally.com. Telephone: (651) 452-4058 or 
(800) 788-4085. Fax: (651) 452-9204. 

Scope of use 

The program was first published in 1992. According to the 
developer. It has been Implemented with general education and 
ELL students in California, Florida, Georgia, Kansas, Michigan, 
Minnesota, Texas, and Virginia. 

Teaching 

The Read Naturally materials come with a teacher’s manual 
that includes the rationale for the program, descriptions of 
the materials needed to implement the program, instructions 



for implementing the program, and sample lesson plans for 
Introducing the program to students. As part of the Intervention, 
students practice reading expository passages until they are 
able to demonstrate improvement in oral language fluency and 
appropriate phrasing and expression. Tutors use vocabulary 
words essential to understanding the passage in a sentence in 
the same way they were used in the passage. Tutors also ask 
questions to stimulate a discussion related to these key words. 

Cost 

Individual Read Naturally materials range In price from $5 to 
$299. The specific needs of the students served will determine 
the materials needed and the cost of implementation. There are 
four levels of Read Naturally materials developed specifically 
for English language learners. The materials for each level cost 
$109. 



One study (Denton, Anthony, Parker, & Hasbrouck, 2004) 
reviewed by the WWC investigated the effects of a modified ver- 
sion of Read Naturally on an English language learner sample. 
Although there was random assignment to treatment groups, 
three students assigned to the control group were reassigned to 
the treatment group, and three students assigned to the treat- 
ment group were reassigned to the control group one week after 
the study had begun (as requested by the participating schools). 
Therefore, this study was determined to be a quasi-experimental 
design that met the WWC evidence standards with reservations. 
In addition, data from three students in the comparison group 
were eliminated from the analysis because of exposure to Read 



Naturally in their classroom, and no data were eliminated from 
analysis in the treatment group. Although this created differential 
attrition rates between the study groups (10% attrition in the 
comparison group and 0% attrition in the treatment group), the 
authors were able to demonstrate post-attrition equivalence 
between groups on the pretest. 

The part of Denton, Anthony, Parker, and Flasbrouck (2004) 
that examined Read Naturally included 60 participants. The 
Read Naturally intervention groups received English language 
pull-out tutoring during the school day in addition to their regular 
English instruction. The control group received only their regular 
English instruction. 



3. Note that this study modified the Read Naturally program. No audio-tapes were used, and the tutors verbaiiy discussed vocabuiary, used a combination 
of fiashcards and prereading strategies (for exampie, preteaching important or chaiienging vocabuiary in reading passages), and did not employ the 
intervention’s approach to progress monitoring. According to the developer, these efforts would have reduced time reading text. 



Read Naturally 



October 5, 2006 



2 




Effectiveness 



The WWC found Read 
Naturally to have no 
discernible effects on 
reading achievement. 
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Findings'* 

The WWC review of interventions for Engiish language iearn- 
ers addresses student outcomes in three domains: reading 
achievement, mathematics achievement, and Engiish language 
development. 

Reading achievement. Denton and colleagues (2004) 
reported, and the WWC confirmed, no statistically significant 
differences between the intervention and comparison groups on 
students’ reading achievement. In addition, the average effect 
size was small and deemed not substantively important. There- 
fore, the one study reviewed showed no discernible effects. 



Rating of effectiveness 

The WWC rates interventions as positive, potentially positive, 
mixed, no discernible effects, potentially negative, or negative. 
The rating of effectiveness takes into account four factors: the 
quality of the research design, the statistical significance of the 
findings, the size of the difference between participants in the 
intervention condition and the comparison condition, and the 
consistency in findings across studies (see the WWC Interven- 
tion Rating Scheme) . 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain the WWC 
computes an average improvement index for each study and 
an average improvement index across studies (see Technical 
Details of WWC-Conducted Computations) . The improvement 
index represents the difference between the percentile rank 
of the average student in the intervention condition versus the 
percentile rank of the average student in the comparison condi- 
tion. Unlike the rating of effectiveness, the improvement index is 
entirely based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 



The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results. The aver- 
age improvement index for reading achievement is 0 percentile 
points across the one study, with a range of -5 to +6 percentile 
points across findings. 

Summary 

The WWC reviewed one study on Read Naturaily. This study met 
WWC standards with reservations. This study found no discern- 
ible effects on reading achievement. The evidence presented in 
this report is limited and may change as new research emerges. 



4. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within class- 
rooms or schools and for multiple comparisons. For an explanation, see the WWC Tutorial on Mismatch . See the Technical Details of WWC Conducted 
Computations for the formulas the WWC used to calculate the statistical significance. In the case of Read Naturally, no corrections for clustering or 
multiple comparisons were needed. 
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RefersncSS Met WWC evidence standards with reservations 

Denton, C. A., Anthony, J. L., Parker, R., & Hasbrouck, J. E. 
(2004). Effects of two tutoring programs on the English read- 
ing development of Spanish-Engllsh bilingual students. The 
Elementary School Journal, 104(A), 289-305. 



Additional sources: 

Denton, C. A. (2000). The efficacy of two English interventions 
in a bilingual education program. Unpublished doctoral dis- 
sertation, Texas A&M University, College Station. 

Ihnot, C. (1992). Read Naturally. St Paul, MN: Read Naturally. 



For more information about specific studies and WWC caicuiations, piease see the WWC Read Naturally 
Technicai Appendices . 
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Appendix 



Appendix A1 Study characteristics: Denton, Anthony, Parker, & Hashrouck, 2004 (quasi-experimentai design) 



Characteristic 


Description 


Study citation 


Denton, C. A., Anthony, J. A., Parker, R., and Hashrouck J. E. (2004). Effects of two tutoring programs on the English reading development of Spanish-English bilingual 
students. The Elementary SchoolJournal, ?04(4), 289-305. 


Participants 


The report by Denton and colleagues covers two studies: one investigates the impacts of Read Naturally and the other investigates the impacts of Read Well. Ninety-three 
Hispanic ELL students (of which 45 were females) who were identified as having difficulty learning to read English participated in one of the two studies. All students who 
participated in both studies identified Spanish as their first language and were bilingual. The participants were in second through fifth grade (2nd = 22; 3rd = 37; 4th = 28; 
5th = 6) and their ages ranged from 7 years to 12 years (average age = 9 years). The 63 students assigned to the Read Naturally study'' were randomly assigned to either 
the treatment or control group. Three students in the control group were exposed to the treatment. These students were not reassigned to the treatment group. Instead, data 
from these students were removed from analysis.^ Additionally, as requested by the participating schools, three students assigned to the control group were reassigned to 
the treatment group, and three students assigned to the treatment group were reassigned to the control group one week after the study had begun. This renders the study a 
quasi-experimentai design. The final sample consisted of 60 students (n = 32 in the treatment group and n = 28 in the control group). 


Setting 


The students attended one of five schools in a Central Texas school district. The district served 13,664 total students, 32% of whom were Hispanic and 56% of whom were 
identified as economically disadvantaged. 


Intervention 


The program occurred during pull-out tutoring sessions during the school day when the participants were not receiving their regular English instruction. Students involved with 
both programs {Read Naturally and Read Well) received an average of 22 tutoring sessions that were 40 minutes in length. The sessions consisted of repeated oral reading 
of connected text, vocabulary and comprehension instruction, and systematic monitoring of progress within the program. The standard Read A/atura/Zy program was modified 
for use with English language learners by adding and extending activities related to vocabulary, decoding, and comprehension (such as, oral discussions of vocabulary and 
comprehension and preteaching important or challenging vocabulary in reading passages). 


Comparison 


The control group received the same regular English education curriculum as the treatment group but did not receive any additional tutoring. 


Primary outcomes 
and measurement 


The study measures in the reading achievement domain included a researcher-developed oral reading assessment^ and three scales from the Woodcock Reading Mastery 
Tests-Revised: Word Identification, Word Attack, and Reading Comprehension. (See Appendix A2 for more detailed descriptions of outcome measures.) 


Teacher training 


Twenty-three undergraduate students studying special education who were enrolled in a class for teaching students with reading difficulties served as tutors. Tutors received 
training and were supervised by a graduate student experienced in Read Naturally. 


1. Students were assigned to one of the two interventions, Read Well or Read Naturally, based on their pretest scores on the Word Attack subtest of the Woodcock Reading Master Tests-Revised (WRMT-R). Students who scored beiow the 
first-grade equivaiency (< 1.0) were assigned to the Read Well sMy, and students whose grade equivaiency score was higher than first grade (> 1.0) were assigned to the Read Naturally s\ur\y. 

2. Because data from three students in the comparison group were eiiminated from the anaiysis and no data were eiiminated from anaiysis in the treatment group, there was differentiai attrition (10% attrition in the comparison group and 
0% attrition in the treatment group). The study did, however, demonstrate post-attrition equivaience. 

3. Data from the researcher-deveioped orai reading assessment were not inciuded in the study. Denton and coiieagues (2004) stated that “iogisticai probiems, some instances of potentiaiiy unreiiabie administration, and missing data 
points resuited in data that is invaiid for anaiyses” (p. 296). 
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Appendix A2 Outcome measures in the reading achievement domain 



Outcome measure 


Description 


Woodcock Reading Mastery 
Tests-Revised (WRMT-R): 
Word Identification subtest 


The Word Identification subtest assesses basis reading skills by having participants read words presented in a list (as cited in Denton et al., 2004). 


WRMT-R; Word 
Attack subtest 


The Word Attack subtest assesses phonemic decoding by having participants read a list of nonsense words aloud (as cited in Denton et al., 2004). 


WRMT-R; Passage 
Comprehension subtest 


The Passage Comprehension subtest uses a cloze format that requires participants to read a passage that has an omitted word. The participants are asked to supply an 
appropriate word to complete the passage that they are reading (as cited in Denton et al., 2004). 
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Appendix A3 Summary of study findings inciuded in the rating for the reading achievement domain^ 



Author’s findings from the study 



Outcome measure 



Study 

sample 



Sample size 
(students) 



Mean outcome 
(standard deviation^) 



Read 

Naturally 

group 



Comparison 

group 



WWC calculations 



Mean difference^ 

{Read Naturally- Effect 

comparison) size’* 



Statistical 
significance^ 
(at a = 0.05) 



Improvement 

index^ 



Denton et al., 2004 (quasi-experimental design)^ 



WRMT-R: Word Identification 


Grades 2-5 


60 


1.12 

(11.64) 


1,75 

(9.65) 


-0.63 


-0.06 


ns 


0 


WRMT-R: Word Attack 


Grades 2-5 


60 


-0,22 

(9.37) 


0.97 

(8.99) 


-1.19 


-0.13 


ns 


-5 


WRMT-R: Passage 
Comprehension 


Grades 2-5 


60 


2,13 

(7,90) 


0,71 

(10.26) 


1.42 


0,16 


ns 


-f6 


Domain average® for reading achievement 










-0,01 


ns 


0 



ns = not statistically significant 



1 . This appendix reports findings considered for the effectiveness rating and the improvement index. 

2. The standard deviation across all students in each group shows how dispersed the participants’ outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes. 

3. Intervention and control group pre- to posttest change scores were used in the study authors’ analyses and in the WWC calculations. Positive differences and effect sizes favor the intervention group; negative differences and effect 
sizes favor the comparison group. 

4. For an explanation of the effect size calculation, please see | H chnical Details of WWC-Conducted Computations . 

5. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. 

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on values 
between -50 and -h 50, with positive numbers denoting favorable results. 

7. The level of statistical significance was reported by the study authors or, where necessary, calculated by the WWC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the 
clustering correction, see the WWC Tutorial on Mismatch . See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Read Naturally, no corrections for 
clustering or multiple comparisons were needed. 

8. This row provides the study average, which in this case is also the domain average. The WWC-computed domain average effect size is a simple average rounded to two decimal places. The domain improvement index is calculated from 
the average effect size. 
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Appendix A4 Read Naturally rating for the reading achievement domain 



The WWC rates interventions as positive, potentiaiiy positive, mixed, no discernibie effects, potentiaiiy negative, or negativeJ 
For the outcome domain of reading achievement, the WWC rated Read Naturally as having no discernibie effects. 



Rating received 

No discernible effects: No affirmative evidence of effects. 

' Criterion 1: None of the studies shows a statistically significant or substantiveiy important effect, either positive or negative. 

Met. The WWC anaiysis found no statistically significant or substantiveiy important effects in this domain. 

Other ratings considered 

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: Two or more studies showing statistically significant positive effects, at least one of which met WWC evidence standards for a strong design. 

Not met. No study that met WWC evidence standards showed statisticaiiy significant positive effects. Further, there was oniy one study, and it did 
not meet WWC standards for a strong design. 

• Criterion 2: No studies showing statistically significant or substantively important negative effects. 

Met. The WWC anaiysis found no statistically significant or substantiveiy important negative effects in this domain. 

Potentially positive effects: Evidence of a positive effect with no overriding contrary evidence. 

• Criterion 1: At ieast one study showing a statistically significant or substantiveiy important positive effect. 

Not met. The WWC anaiysis found no statistically significant or substantiveiy important positive effects in this domain. 

• Criterion 2: No studies showing a statisticaiiy significant or substantively important negative effect and fewer or the same number of studies showing indeterminate 
effects than showing statisticaiiy significant or substantively important positive effects. 

Not met. One study showed indeterminate effects and no study showed positive effects. 

Mixed effects: Evidence of inconsistent effects as demonstrated through either of the following criteria. 

• Criterion 1: At ieast one study showing a statistically significant or substantiveiy important positive effect, and at ieast one study showing a statistically significant 
or substantiveiy important negative effect, but no more such studies than the number showing a statisticaiiy significant or substantiveiy important positive effect. 

Not met. The WWC anaiysis found no statistically significant or substantiveiy important positive or negative effects in this domain. 

• Criterion 2: At least one study showing a statistically significant or substantively important effect, and more studies showing an indeterminate effect than showing 
a statistically significant or substantively important effect. 

Not met. The WWC analysis found no statistically significant or substantively important effects in this domain. 



1 . For rating purposes, the WWC considers the statistical signiticance of individual outcomes and the domain level effects. The WWC also considers the size of the domain level effects for ratings of potentially positive or potentially nega- 
tive effects. See the WWC Intervention Rating Scheme for a complete description. 



(continued) 
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Appendix A4 Read Naturally rating for the reading achievement domain (continued) 



Potentially negative effects: Evidence of a negative effect with no overriding contrary evidence. 

• Criterion 1: At ieast one study showing a statistically significant or substantiveiy important negative effect. 

Not met. The WWC anaiysis found no statistically significant or substantiveiy important negative effects in this domain. 

• Criterion 2: No studies showing a statisticaiiy significant or substantively important positive effect. Or, more studies showing statisticaily significant or substantively 
important negative effects than showing statisticaiiy significant or substantiveiy important positive effects. 

Not met. The WWC anaiysis found no statistically significant or substantiveiy important negative or positive effects in this domain. 

Negative effects: Strong evidence of a negative effect with no overriding contrary evidence. 

Criterion 1: Two or more studies showing statistically significant negative effects, at ieast one of which is based on a strong design. 

Not met. No study that met WWC evidence standards showed statistically significant negative effects. Further, there was oniy one study, and it did 
not meet WWC standards for a strong design. 

• Criterion 2: No studies showing statistically significant or substantively important positive effects. 

Met. The WWC anaiysis found no statistically significant or substantiveiy important positive effects in this domain. 
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